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Summary 

The  DARPA  AWARE  Wide  Field  of  View  program  successfully  demonstrated  that  a  common  microcamera 
platform  can  be  used  to  quickly  and  effectively  provision  wide  field  of  view  high  resolution  cameras  with 
pixel  counts  between  0.25  and  10  gigapixels.  The  array  camera  architecture  uniquely  allows  high  depth 
of  field  and  high  dynamic  range  imaging  using  independent  parallel  control  of  the  microcamera  array. 
Moving  forward,  the  array  camera  approach  will  be  applied  to  diverse  applications  in  visible  and  infrared 
imaging. 

AWARE  produced  four  gigapixel  camera  systems:  the  AWARE  2  with  a  40  prad  instantaneous-field-of- 
view  (IFOV)  and  0.3  to  1  gigapixel,  the  AWARE  10  with  25  prad  IFOV  and  1  to  4  gigapixels,  the  AWARE  40 
with  12  prad  IFOV  and  3  gigapixels,  and  the  MOAC  with  17  prad  and  8  to  11  gigapixels.  The  prototypes 
of  the  AWARE  2  and  40  cameras  are  shown  below.  The  project  has  realized  DARPA' s  goal  of  achieving  a 
scalable,  modular,  and  economic  architecture  including  a  design  methodology  and  completed 
prototypes.  Furthermore,  the  project  has  already  demonstrated  surveillance  capability  far  superior  to 
existing  shipboard  naval  survelliance  systems  by  reliable  identification  of  potential  threat  watercraft  as 
small  as  7  m  at  a  distance  of  6  km  over  a  100°  field-of-view  (FOV)  with  the  AWARE  10  camera.  While  this 
initial  demonstration  showed  the  great  potential  applications  of  the  DARPA-sponsored  technology  to 
provide  the  resolution  of  a  telescope  over  a  wide-field,  further  demonstrations  and  adaptations  would 
speed  the  adoption  and  ease  the  transition  of  the  technology  to  the  armed  services.  The  four 
architectures  that  have  been  constructed  are  adaptable  to  urgent  and  unsatisfied  applications  including 
wide-field  high-resolution  aerial  reconnaissance,  space  situational  awareness,  terrestrial  perimeter 
surveillance,  as  well  as  the  naval  threat  identification  applications  already  explored. 


AWARE  2  folded  camera,  40  jurad 
IFOV,  50°  by  30°  FOV,  fits  in  a 
commercial  airline  carry-on  bag. 


AWARE  40  camera 
12  (irad  IFOV,  30°  FOV 


Requirements  and  Specifications 

The  original  program  considered  construction  of  a  50  gigapixel  camera.  Design  analysis  and 
optical  testing  confirmed  the  feasibility  of  such  a  camera,  but  budget  constraints  limited  as  built 
systems  to  10  gigapixels.  Specifically,  an  array  of  3  AWARE  40  cameras  was  shown  to  be  capable 
of  meeting  the  demonstration  specifications  listed  below.  The  performance  of  this  system  is 
reported  in  Patrick  Llull,  Lauren  Bange,  Zachary  Phillips,  Kyle  Davis,  Daniel  L.  Marks,  and  David  J. 
Brady,  "Characterization  of  the  AWARE  40  wide-field-of-view  visible  imager,"  Optica  2,  1086-1089 
(2015) 


MOSAIC  Phase  II  demonstration 

Integrated  optical  and  opto-electronic 
prototype  volume 

0.25  cubic  meter 

Prototype  mass 

90  kg 

Field  of  view  (FOV) 

100  by  60  degree 

Instantaneous  field  of  view  (ifov) 

16.6  microradian  cone  angle 

Image  size 

10  gigapixel 

Image  type 

RGB  color 

Operating  modes 

Full  frame  snapshot  and  live  streaming  interface 

Image  quality 

Comparable  to  600  mm  Canon  SLR  baseline 

Motion  and  turbulence  artifacts 

Component  level  demonstration 

Frame  rate 

6  Hz 

Frame  buffer 

>30 

Image  formation  latency 

<1  second 

The  AWARE  program  was  pursued  in  two  phases,  in  the  first  phase  a  gigapixel  camera  was  constructed 
and  tested  with  results  reported  in  Brady,  D.  J.,  et  al.  (2012).  "Multiscale  gigapixel  photography."  Nature 
486(7403):  386-389. 

The  second  phase  used  next  generation  glass  optics  and  an  improved  processor  architecture. 

Project  tasks  included  system  design,  optical  design,  electronic  system  design  and  prototype 
construction  and  testing. 

Results  for  these  tasks  were  reported  in  peer  reviewed  publications. 


Publications  considering  limits  of  AWARE  system  performance  include 

Brady,  D.  J.,  et  al.  (2013).  Petapixel  Photography  and  the  Limits  of  Camera  Information  Capacity. 
Computational  Imaging  Xi.  C.  A.  Bouman,  I.  Poliak  and  P.  J.  Wolfe.  8657. 

Marks,  D.  L.  and  D.  J.  Brady  (2011).  "Close-up  imaging  using  microcamera  arrays  for  focal  plane 
synthesis."  Optical  Engineering  50(3). 

Marks,  D.  L.  and  D.  J.  Brady  (2013).  "Wide  field  astronomical  multiscale  cameras."  Astronomical  Journal 
145(5). 


Publications  reporting  results  from  camera  systems  tests  include 

Brady,  D.  J.,  et  al.  (2012).  "Multiscale  gigapixel  photography."  Nature  486(7403):  386-389. 

Llull,  P.,  et  al.  (2015).  "Characterization  of  the  AWARE  40  wide-field-of-view  visible  imager."  Optica 
2(12):  1086-1089. 

Marks,  D.  L.,  et  al.  (2014).  "Characterization  of  the  AWARE  10  two-gigapixel  wide-field-of-view  visible 
imager."  Applied  Optics  53(13):  C54-C63. 

Publications  describing  the  image  processing  pipeline  include 

Golish,  D.  R.,  et  al.  (2012).  "Development  of  a  scalable  image  formation  pipeline  for  multiscale  gigapixel 
photography."  Optics  Express  20(20):  22048-22062. 

Iliopoulos,  A.  S.,  et  al.  (2013).  Big  Snapshot  Stitching  with  Scarce  Overlap,  2013  IEEE  Conference  on  High 
Performance  Extreme  Computing 

Marks,  D.  L.,  et  al.  (2015).  "Feedback  stitching  for  gigapixel  video."  Journal  of  Electronic  Imaging  24(6). 

Nakamura,  T.,  et  al.  (2013).  "Autofocus  for  a  multiscale  gigapixel  camera."  Applied  Optics  52(33):  8146- 
8153. 


Publications  describing  optical  system  design  and  performance  include 

Marks,  D.  L.,  et  al.  (2011).  "Microcamera  aperture  scale  in  monocentric  gigapixel  cameras."  Applied 
Optics  50(30):  5824-5833. 

Tremblay,  E.  J.,  et  al.  (2012).  "Design  and  scaling  of  monocentric  multiscale  imagers."  Applied  Optics 
51(20):  4691-4702. 

Zheng,  N.,  et  al.  (2012).  "Computer  experiment  and  global  optimization  of  layered  monocentric  lens 
systems."  Optik  123(14):  1249-1259. 


Publications  describing  optomechanical  design  and  testing  include 

Marks,  D.  L.,  et  al.  (2012).  "Engineering  a  gigapixel  monocentric  multiscale  camera."  Optical  Engineering 
51(8). 

Kittle,  D.  S.,  et  al.  (2013).  Automated  calibration  and  optical  testing  of  the  AWARE-2  gigapixel  multiscale 
camera.  Digital  Photography  lx.  N.  Sampat  and  S.  Battiato.  8660. 

Kittle,  D.  S.,  et  al.  (2013).  "A  testbed  for  wide-field,  high-resolution,  gigapixel-class  cameras."  Review  of 
Scientific  Instruments  84(5). 

Son,  H.  S.,  et  al.  (2013).  "Optomechanical  design  of  multiscale  gigapixel  digital  camera."  Applied  Optics 
52(8):  1541-1549. 

Son,  H.  S.,  et  al.  (2013).  "Oversampled  triangulation  of  AWARE-10  monocentric  ball  lens  using  an  auto- 
stigmatic  microscope."  Optics  Express  21(19):  22206-22214. 

Son,  H.  S.,  et  al.  (2011).  "Design  of  a  spherical  focal  surface  using  close-packed  relay  optics."  Optics 
Express  19(17):  16132-16138. 

Son,  H.  S.,  et  al.  (2013).  "Design  of  a  spherical  focal  surface  using  close-packed  relay  optics:  erratum." 
Optics  Express  21(22):  27284-27285. 

Son,  H.  S.,  et  al.  (2013).  Alignment  and  assembly  strategies  for  AWARE-10  gigapixel-scale  cameras. 
Optomechanical  Engineering  2013.  A.  E.  Hatheway.  8836. 

Youn,  S.  H.,  et  al.  (2013).  Efficient  testing  methodologies  for  microcameras  in  a  gigapixel  imaging  system. 
Optical  Measurement  Systems  for  Industrial  Inspection  Viii.  P.  H.  Lehmann,  W.  Osten  and  A.  Albertazzi. 
8788. 

Youn,  S.  H.,  et  al.  (2014).  "Optical  performance  test  and  validation  of  microcameras  in  multiscale, 
gigapixel  imagers."  Optics  Express  22(3):  3712-3723. 


Technology  transition  was  also  a  critical  component  of  the  AWARE  program.  DARPA  supported  a  follow 
project  with  Applied  Quantum  Technologies,  in  Durham,  NC  to  support  continued  operation  of  the 
AWARE  cameras  and  development  of  next  generation  systems.  As  part  of  this  project,  additional  tests 
on  at  the  Naval  Research  Laboratory  testing  sight  on  the  Chesapeake  Bay  were  conducted  in 
collaboration  with  J.  M.  Nichols,  C.  C.  Olson,  K.  P.  Judd,  K.  Novak  of  NRL.  The  results  of  this  study  will 
be  submitted  in  a  future  peer  reviewed  publication.  A  draft  version  of  the  study  is  included  below. 
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FIG.  1.  (a)  Basic  AWARE  camera  architecture  whereby  a  single,  spherical  objective  lens  distributes  the  incident  scene  across 
an  array  of  visible  micro-cameras,  (b)  The  original  AWARE  camera  system  occupying  a  volume  >  0.15  m3  and  consuming 
>  1  kW  of  power,  (c)  The  new  AWARE  camera  measuring  0.305m  x  0.228m  x  0.762m  (0.053  m3),  weighing  19kg  and  consuming 
roughly  100  Watts  of  power 


I.  BACKGROUND 


In  a  prior  work,  [1],  we  documented  the  design  and  performance  of  a  new  camera  architecture  developed  as  part 
of  the  DARPA  Advanced  Wide  FOV  Architectures  for  Image  Reconstruction  and  Exploitation  program  (referred 
to  herein  as  AWARE).  The  goal  of  AWARE  is  to  simultaneously  provide  both  wide  field-of-view  (FOV)  imaging 
capability  and  the  resolution  required  to  detect  and  identify  distant  targets.  This  was  accomplished  by  leveraging  the 
power  of  a  large  number  of  small,  cheap  focal  plane  arrays  to  produce  a  single,  high-resolution  image  over  a  wide  FOV. 
Specifically,  the  previous  system  afforded  the  user  a  100°  by  60°  field-of-view  with  an  instantaneous  FOV  of  25/irad; 
a  resolution  that  meets  many  of  the  Navy’s  requirements  for  identifying  targets  at  range.  These  twin  objectives  were 
met  by  distributing  light  from  a  single  aperture  over  a  large  number  of  micro-cameras  using  a  spherical  objective  lens 
as  shown  in  Figure  (la).  The  resulting  architecture  has  the  advantage  of  being  highly  modular  and  easily  scalable 
to  larger  FOVs  as  one  can  simply  populate  the  camera  housing  with  additional  micro-cameras  as  dictated  by  the 
application. 

While  the  system  simultaneously  addresses  multiple  (competing)  imaging  requirements,  the  challenge  lies  in  mini¬ 
mizing  the  physical  footprint  of  the  system.  The  previous  architecture  occupied  a  volume  of  >  0.15  m3,  and  consumed 
>  1  kW  of  power;  this  system  is  shown  in  Figure  (lb).  Over  the  last  two  years  we  have  therefore  worked  to  maintain 
or  improve  the  system  optical  performance  (e.g.,  target  identification),  while  significantly  reducing  the  system  size, 
weight  and  power.  The  end  result  is  shown  in  Figure  (lc)  and  achieves  both  of  these  aims. 

In  what  follows  we  document  and  compare  this  new  generation  of  camera  with  the  previous  version.  We  then  repeat 
the  maritime  target  identification  experiment  performed  in  reference  [1]  and  provide  a  comparison  of  the  resulting 
range  performance  curves.  The  results  obtained  for  the  new,  smaller  system  show  significant  improvement  in  range 
performance  over  the  previous  incarnation. 
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FIG.  2.  Image  processing  pipeline  for  a  single  micro-camera 


II.  IMPROVED  CAMERA  ARCHITECTURE 


In  addition  to  minimizing  system  size,  weight,  and  power  (SWaP),  two  major  areas  were  identified  for  improvement 
over  the  previous  AWARE  camera:  micro-camera  focus  and  optimization  of  the  image  processing  pipeline.  Regarding 
the  former,  significant  improvements  were  obtained  by  adding  hardware-based  auto-focus  to  this  system.  Specifically, 
auto- focus  was  performed  by  adding  a  contrast  detection  algorithm  onto  the  field-programmable  gate  arrays  (FPGA) 
that  managed  the  data  capture  and  controlled  the  focus  positioner.  This  allowed  for  automated  focusing  of  each 
individual  micro-camera  which  was  faster  and  more  reliable  than  the  manual  focusing  done  in  the  earlier  experiment 
documented  in  [1]. 

Additional  improvements  were  made  to  both  the  image  processing  pipeline  as  well  as  the  data  routing  architecture 
required  to  move  the  large  volume  of  data  reliably  and  efficiently  from  camera  to  computer.  Both  of  these  areas  are 
discussed  next. 


A.  Image  Processing  Improvements 

The  overall  data  processing  pipeline  for  the  new  system  is  shown  schematically  in  Figure  (2).  By  improving 
the  image  processing  pipeline,  the  detection  task  was  made  easier  by  providing  higher  fidelity  images  with  better 
calibration.  The  image  processing  pipeline  was  optimized  by  mapping  the  12-bit  sensor  data  non-linear ly  onto  the 
8-bit  data  flow  that  the  FPGA  required  for  data  transmission.  A  look-up  table  (LUT)  was  generated  with  a  power 
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law  of  0.5, 

Pcomp  =  Praw  (1) 

Where  Praw  is  the  linear  12-bit  sensor  pixel  data  and  PCOmp  is  the  compressed  pixel  data  which  is  then  quantized  at 
8-bits.  The  end  result  of  this  was  to  match  the  quantization  spacing  of  the  measured  data  with  the  Poisson  noise  of 
the  measured  data  (which  has  a  similar  square  root  dependence).  The  original  AWARE  system  performed  this  12  to 
8  bit  conversion  by  dropping  the  last  4  bits  of  the  image  sensor  data.  This  8-bit  data  was  then  sent  packetized  and 
transmitted  by  the  FPGA  to  the  computer  cluster  where  it  was  stored  in  local  memory  and  then  either  written  to 
disk,  sent  to  the  rendering  system  if  needed,  or  temporarily  cached. 

The  image  rendering  task  of  assembling  one  zoomable  image  from  an  array  of  image  sensors  followed  the  same 
geometrical  mapping  developed  in  [1].  Changes  in  the  rendering  pipeline  focused  mostly  on  the  radiometric  processing 
of  the  sensor  data.  After  linearizing  the  sensor  data  based  on  the  LUT  applied  at  the  FPGA,  a  white  balance  was 
applied  by  scaling  the  sensor  data  that  corresponded  to  the  Bayer  color  filter  channels  (one  red,  two  green,  and 
one  blue)  on  the  sensor.  Each  image  sensor  had  its  white  balance  scaled  differently  based  on  a  flat  field  calibration 
measurement  done  by  placing  a  diffuser  in  front  of  the  entire  camera  during  calibration.  Demosaicing  was  performed 
efficiently  in  the  OpenGL  shader  by  using  kernels  similar  to  the  approach  of  Malvar  et  al  [2] . 

Vignetting  correction  was  accomplished  by  performing  a  pixel-by-pixel  scaling  of  the  demosaiced  imagery  based  on 
a  flat  field  measurement  captured  during  calibration.  The  parameters  of  the  vignetting  included  the  center  point  of 
the  radial  map,  and  three  10th  order  polynomial  fits  describing  the  radial  roll-off  for  each  color  channel.  After  this 
vignetting  correction,  a  3x3  matrix  multiplication  converts  the  sensor  RGB  colors  to  the  standard  sRGB  to  allow 
spectrally  consistent  colors  on  monitors  for  viewing.  Finally,  the  images  from  each  sensor  are  corrected  based  on  the 
exposure  time  used  and  the  calibrated  sensitivity  of  each  image  sensor,  again  based  on  the  flat  field  calibration,  to 
allow  viewing  of  imagery  from  different  sensors  on  the  same  display.  A  global  tone-mapping  is  then  controlled  at  the 
rendering  system  by  adjusting  brightness  and  contrast  to  optimize  the  display  of  imagery.  The  image  resizing  and 
warping  is  done  to  map  the  correct  data  from  the  sensor  array  to  render  the  view  of  interest  to  the  user.  A  joystick 
provided  pan,  zoom,  and  tilt  navigation  through  the  field-of-view  with  buttons  for  pausing  and  advancing  backwards 
and  forwards  in  time. 


B.  Improvements  in  Data  Routing  Architecture 

One  of  the  chief  problems  encountered  when  developing  systems  with  very  large  formats  is  physically  moving  the 
data  from  camera  to  display.  In  this  section  we  discuss  the  changes  in  the  data  routing  architecture  and  operating 
system  to  improve  performance.  These  changes  including  a  new  data  routing  paradigm,  advanced  image  caching 
architecture,  and  better  prediction  algorithms  for  real-time  data  access.  These  changes  were  driven  by  the  need  to 
make  a  more  compelling  user  interface  through  decreased  latency,  improved  frame  rate  and  a  more  responsive  user 
feedback. 

For  the  AWARE  camera  discussed  in  this  paper,  this  architecture  is  able  to  provide  multiple  interactive  user 
interfaces  to  a  single  camera  with  image  updates  approaching  5.6  frames  per  second  with  less  than  a  second  latency 
from  a  real-world  event.  This  is  significantly  more  responsive  that  our  previous  implementation  while  maintaining 
the  flexibility  to  scale  the  system  to  an  arbitrary  number  of  sensors  without  significant  degradation  in  performance. 
Figure  (3)  outlines  the  new  system  architecture: 

The  original  architecture  implemented  a  state  machine  for  rendering  individual  frames.  For  each  frame,  the  rendering 
system  would  calculate  the  scale  and  a  region  of  interest  required  by  each  camera  to  render  the  scene  to  the  display. 
This  list  of  image  requests  would  then  get  transferred  to  the  embedded  FPGAs,  on  the  system  which  would  resize 
and  crop  the  raw  image  data  before  sending  a  final  image  of  the  appropriate  resolution  and  scale.  In  the  interim,  the 
user  would  be  able  to  interact  with  the  current  frame  until  the  requested  one  arrives.  This  approach  minimizes  the 
number  of  pixels  transmitted  to  the  rendering  system,  but  its  sequential  nature  limited  the  screen  update  rate  to  two 
frames  per  second  and  leads  to  significant  lag  before  an  event  would  appear  on  screen.  In  addition,  the  embedded 
hardware  had  very  limited  storage  capacity,  so  this  precluded  storage  of  video  sequences  more  that  about  30  frames 
in  size. 

The  current  camera  architecture  was  designed  for  real-time  video  broadcast  applications  and  uses  a  parallel  ac¬ 
quisition  and  rendering  architecture  to  minimize  latency  and  provide  full-resolution  video  capture  support.  In  this 
approach  we  separate  the  rendering  pipeline  from  the  data  acquisition  pipeline  through  the  use  of  a  computational 
cluster  where  each  node  receives  data  from  up  to  six  microcameras  and  handles  data  requests  from  the  rendering 
system.  A  memory  manager  is  used  to  cache  hundreds  or  thousands  of  frames  based  on  image  resolution  while  storing 
data  to  one  or  more  solid  state  drives.  Images  requested  from  the  rendering  system  are  read  from  the  cache  or  read 
from  disk  then  resized  to  the  appropriate  scale  before  being  sent.  Using  the  computational  cluster  as  a  middle  layer 
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FIG.  3.  Data  routing  architecture  for  the  new  AWARE  camera 


provides  additional  memory  for  caching,  provides  a  common  interface  to  multiple  rendering  systems,  and  provides 
scalable  storage  for  gigapixel-scale  data  streams. 

The  rendering  system  uses  an  OpenGL  framework  similar  to  the  one  outlined  in  the  previous  paper  to  generate 
an  interactive  user  interface.  The  raw  bayer  image  data  is  demosaiced  with  GL  shaders  and  mapped  to  a  polygon 
mesh  which  is  warped  based  on  the  sensor  model  as  described  in  the  section  on  the  image  processing  pipeline.  To 
increase  the  rendering  update  rate,  a  predictive  algorithm  is  used  to  pre-fetch  images  from  the  computer  cluster  that 
are  likely  to  be  needed  in  upcoming  frames.  The  aggressiveness  of  prediction  is  scaled  based  on  available  bandwidth 
and  the  computational  resources  of  the  cluster.  In  addition  to  the  interface  described  in  the  previous  paper,  the 
current  rendering  application  also  provides  a  joystick  interface  for  intuitive  interaction  with  the  data. 


C.  Camera  Specifications  and  Testing 

As  we  mentioned  in  section  (I),  one  of  the  main  objectives  of  this  work  was  to  reduce  SWaP  while  maintaining 
optical  imaging  performance.  The  volume  of  the  new  system  was  approximately  one  third  of  the  system  described 
in  [1]  and  less  than  half  the  weight  (see  again  Figure  1  and  the  surrounding  discussion).  Identical  optics  (spherical 
objective  lens)  were  used  in  both  old  and  new  AWARE  camera  systems  although  the  newer  system  possessed  a 
restricted  field-of-view;  obtained  by  reducing  the  number  of  micro-cameras  to  allow  a  more  compact  camera  head 
package.  This  trade-off  maintains  the  system  resolution  on  the  horizon  (as  compared  to  the  older  system)  however 
would  prohibit  the  detection  or  airborne  targets  that  are  beyond  the  13°  look-up  angle.  For  maritime  identification 
(our  intended  application)  this  trade-off  (field-of-view  for  size,  weight,  and  power)  is  appropriate. 

The  camera  head  is  shown  in  Figure  (lc)  and  has  a  305  mm  x  228  mm  x  762  mm  footprint  and  weighs  19  kilograms. 
Each  camera  required  a  100  watt  power  supply  and  a  water  filled  heat  exchanger  to  dissipate  heat  generated  by  the 
4  FPGAs  inside  each  camera.  The  original  camera  used  2  kilowatts  due  to  the  increased  number  of  image  sensors 
(158  vs  30),  however  even  correcting  for  the  decreased  number  of  cameras  in  this  version  of  the  system  the  power 
consumption  is  nearly  an  order  of  magnitude  improvement. 

All  data  communications  between  the  camera  head  and  the  rendering  computers  was  accomplished  via  single¬ 
mode  SFP+  10  Gigabits/second  optical  fiber  transceivers.  These  signals  were  multiplexed  using  multi- wavelength 
transceivers  to  allow  the  entire  50  Gigabits/second  of  data  to  be  relayed  back  to  the  computer  cluster  on  2  single-model 
fibers. 

During  field  testing  (see  section  III)  the  video  from  the  camera  was  provided  to  an  operator  with  a  latency  of 
under  a  second.  A  zoomed  out  view  of  the  field-of-view  is  shown  in  Figure  (4),  approximately  53°  x  13.6°  with  an 
instantaneous  field-of-view  of  25  micro-radians.  Two  AWARE  cameras  were  used  to  speed  up  testing  by  allowing 
multiple  trials  to  be  run  simultaneously  using  different  regions  of  the  full  180°  FOV. 

An  example  still  frame  from  the  test  is  shown  in  Figure  4.  One  of  the  camera  sensors  was  malfunctioning  that 
day  which  created  the  white  circle  in  the  lower-left  corner.  Each  camera  had  30  micro-cameras  oriented  in  3  rows  of 
11,  10,  and  9  micro-cameras  each.  A  zoomed  in  view  from  the  top  image  is  shown  at  the  bottom  of  Figure  (4)  and 
shows  the  presence  of  an  unknown  watercraft.  These  images  were  both  acquired  using  an  external  video  recorder  of 
the  HDMI  output  from  the  rendering  computer.  The  stitching  model  used  was  created  during  laboratory  testing  of 
the  camera  and  hence  there  are  minor  stitching  errors  in  the  image.  The  flat  field  calibration  is  unable  to  exactly 
calibrate  the  color  matching  between  sensors  due  to  variances  in  the  spectral  response  of  each  imaging  sensor  thus 
there  are  still  minor  color  fluctuations  between  the  micro-cameras. 


FIG.  4.  Full  panorama  53°  by  13.6°  image  as  displayed  to  the  user  with  a  box  centered  on  a  potential  target  (left)  and  a 
close-up  of  the  target  displayed.  The  camera  is  positioned  at  the  end  of  a  pier  looking  across  the  Chesapeake  Bay  toward 
Tighlman  Island  (barely  visible  on  horizon). 


III.  RANGE  PERFORMANCE 


In  this  section  we  briefly  review  the  range  performance  experiment  and  compare  the  new  camera  results  to  those 
obtained  previously.  Specifically,  our  focus  is  on  assessing  the  range  at  which  a  typical  observer  can  correctly  identify 
a  given  target  with  70%  accuracy.  This  is  a  frequently- used  criteria  in  assessing  the  relative  performance  of  camera 
systems  for  maritime  applications.  In  what  follows  we  document  the  results  obtained  by  from  a  maritime  target 
identification  experiment  performed  at  the  Naval  Research  Laboratory,  Chesapeake  Bay  Detachment.  The  target  set, 
camera  location,  and  relevant  ranges  were  retained  from  our  prior  work  ([1])  in  order  to  make  the  comparison  as 
meaningful  as  possible.  In  fact,  the  experiments  were  performed  over  the  same  daylight  hours  and  nearly  the  exact 
day  of  the  year  as  the  prior  work. 

The  experiment  consisted  of  placing  one  of  three  different  target  ships  at  a  specific  range  and  allowing  a  series  of 
observers  to  attempt  to  identify  the  ship  based  on  video  imagery  displayed  on  a  computer  monitor.  The  ships  chosen 
were  of  a  similar  characteristic  dimension  in  order  to  make  the  task  more  challenging.  Specifically,  the  target  ships 
(and  their  characteristic  dimension)  used  in  this  study  were:  Duck  boat  (4m),  Fishing  boat  (7m),  and  a  Crab  boat 
(10m).  Each  of  these  ships  is  shown  in  Fig.  (5) 

The  computer  monitors  on  which  the  imagery  was  displayed  were  housed  in  a  large,  climate  controlled  trailer.  Once 
placed  at  range,  the  ships  remained  stationary  and  were  always  oriented  broadside  to  the  camera  position  so  that 
the  largest  aspect  was  visible  to  the  observer.  Each  observer  was  called  separately  into  the  trailer  and  given  three 
minutes  to  search  the  image  and  record  a  response.  Observers  could  respond  with  the  ship(s)  identified,  report  that 
no  identification  was  possible,  or  report  that  no  target  was  in  the  scene. 

It  is  also  worth  noting  that  ambient  maritime  traffic  created  a  number  of  false  targets,  or  “clutter”,  during  the 
testing.  For  any  given  trial,  a  number  of  other  ships  of  similar  size  and  type  were  in  the  FOV.  While  this  created  a 
challenging  environment,  it  also  effectively  prevented  the  observers  from  simply  guessing,  or  from  using  a  “process  of 
elimination”  during  trials  in  which  multiple  ships  were  in  the  scene.  This  is  of  particular  importance  in  helping  to 
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FIG.  5.  Ships  used  in  the  maritime  target  identification  experiment. 


justify  the  assumption  of  “independent  trials”  used  in  the  next  section  (see  section  III  A).  This  same  degree  of  clutter 
was  present  in  our  prior  AWARE  camera  experiments. 


A.  Estimation  Approach 


The  first  step  in  any  estimation  procedure  is  to  form  a  model  for  the  observed  data.  To  this  end  we  make  use  of 
the  same  estimation  procedure  outlined  in  [1].  To  briefly  summarize,  we  assume  the  range  performance  model 


P(r,  s ) 


1 

1  +  exp(aisr  -  a2s)  ’ 


s  —  1,2,3 


(2) 


for  target  boat  s  at  continuous  range  r.  In  what  follows  we  use  8  =  1  (duck  boat),  8  =  2  (fishing  boat),  and  8  =  3 
(crab  boat).  The  parameters  a\s  and  a2s  govern  the  rate  at  which  identification  probabilities  decay  from  unity  to  zero 
and  the  range  at  which  this  transition  occurs,  respectively.  The  model  (2)  is  simple,  flexible,  and  accurately  captures 
observer  performance  for  a  wide  range  of  target  detection  experiments.  Moreover,  it  allows  us  to  use  all  available 
range  data  to  estimate  only  two  parameters  per  ship. 

Given  the  model  (2)  for  describing  identification  probability  with  range,  and  assuming  all  observer  trials  are  inde¬ 
pendent,  the  model  for  the  observer  responses  is  given  by  the  multinomial  probability  distribution 


R 


/Ks(ks(Vj)|as)  =  F  — —  Njs' 


hi  “  ks(rj))-ks(rj)- 


1 


1  +  exp(alsrj  -  a2s) 


1  +  cxp(alsr;-  -  a2s) 
k.-(rj) 


l-ks(rj) 


(3) 


where  Njs  are  the  number  of  trials  conducted  for  ship  class  s  at  range  rj,  j  =  1  •  •  •  R,  and  ks{rj)  are  the  number  of 
those  trials  in  which  ship  s  was  correctly  identified.  Given  the  data,  ks(rj ),  the  goal  is  to  estimate  the  parameters 

&1 S5  ^2  S' 

As  in  or  prior  work  we  take  a  Bayesian  approach  where  we  treat  the  parameters  ais,  a2s  as  random  variables 
Ais,  A2s.  Our  goal  is  to  estimate  the  posterior  probability  distribution  functions  (PDFs)  associated  with  each  of 
these  random  variables.  That  is  to  say,  given  the  ks  we  estimate  the  posterior  PDFs  /ais(&is),  fA2s(a‘2s )  and  then 
take  the  distribution  mean  as  the  final  value  for  each  of  the  parameters.  We  again  use  the  Markov  Chain  Monte 
Carlo  (MCMC)  algorithm  [3],  which  allows  us  to  build  a  first  order  Markov  chain  for  each  parameter  (and  for  each 
ship  class)  yielding  M  sample  values  ai8(j),  a2s(j )  j  =  1  •  •  •  M.  For  a  properly  formed  chain  (see  [4]  for  details), 
the  resulting  M— vectors  are  comprised  of  samples  from  fAls{ais),  /A2s(a2s)-  Our  final  parameter  estimates  are  then 
simply  dis  =  J2jLi  ai s(j)  and  &2s  =  ja  Y^jLi  a2s{j )  fc>r  each  of  the  ship  classes,  8.  These  parameters  then  define 
the  range  performance  via  (2). 

The  chains  are  initialized  with  samples  drawn  from  the  uniform  prior  PDFs  and  are  allowed  some  number  of  “burn- 
in”  iterations  B  to  converge  to  a  stationary  sequence  of  values.  Values  for  subsequent  iterations  are  then  saved  as 
samples  from  the  posterior.  Moreover,  these  samples  can  be  used  to  estimate  credible  intervals  (the  Bayesian  analogy 
of  a  confidence  interval)  by  simply  computing  the  upper  and  lower  bounds  on  the  central  95%  of  parameter  values 
and  the  associated  detection  probabilities.  Estimated  range  performance  curves  will  be  presented  in  the  next  section. 


We  note  that  this  estimator  of  range  performance  is  in  contrast  to  the  historical  approach  (see  e.g.,  [5])  whereby 
probability  of  identification  is  estimated  at  each  range  separately  and  independently  of  data  at  other  ranges.  Specifi¬ 
cally,  the  historical  approach  provides  the  maximum  likelihood  estimate  (MLE)  at  each  range  via  [4] 


p(rhs) 


Mr?) 

N-  ’ 

iV  JS 


3  =  1  -R, 


(4) 


where  R  is  again  the  number  of  ranges  over  which  data  is  acquired.  While  this  approach  was  demonstrated  conclusively 
to  produce  poor  results  in  [6],  the  MLEs  at  each  individual  range  will  also  be  presented  in  the  next  section  for 
comparison  purposes. 


B.  Range  Performance  Results  and  Discussion 

Based  on  the  observer  data,  we  used  the  MCMC  algorithm  to  estimate  the  range  performance  parameters  for  each  of 
the  three  ships.  These  results  are  displayed  in  the  bottom  row  of  Fig.  (6)  while  the  top  row  gives  our  prior  results  from 
the  older  AWARE  camera  system.  The  range  performance  curve  associated  with  the  maximum  a  posterior  estimate  is 
given  by  the  solid  line  while  the  95%  credible  intervals  are  given  by  the  dashed  lines  enveloping  the  estimated  curve. 
For  comparison  purposes  we  have  also  plotted  the  MLEs,  given  by  Eq.  (4),  associated  with  a  given  range.  Finally, 
each  curve  also  is  annotated  with  the  range  at  which  70%  probability  of  identification  was  obtained.  The  70%  mark 
is  frequently  used  in  specifying  requirements  for  EO/IR  systems. 


Range  (m) 


FIG.  6.  Estimated  range  performance  curves  obtained  using  the  MCMC  algorithm.  The  top  row  shows  the  results  of  the  prior 
AWARE  camera  (taken  directly  from  [1]  while  the  bottom  row  gives  the  results  for  the  new  system  documented  in  this  work. 
The  maximum  a  posterior  curve  is  given  by  the  solid  line  while  the  associated  95%  credible  interval  is  given  by  the  dashed 
line.  Also  given  are  the  maximum  likelihood  estimates  at  each  range,  denoted  by  crosses  in  the  figure.  The  estimated  70% 
probability  of  identification  range  is  highlighted  with  an  asterisk. 

For  each  of  the  classes  of  ship  tested,  performance  was  improved  over  our  prior  experiments.  The  improvement  was 
most  dramatic  for  both  the  duck  boat  and  fishing  boat.  Both  the  duck  boat  and  fishing  boat  improved  by  several 
kilometers  their  70%  identification  points  which,  in  this  work,  were  found  to  be  7200m  and  9500m  respectively.  This 
owes  at  least  in  part  to  the  improved  optics  which  significantly  reduced  the  number  of  false  classifications  (i.e.,  crab 
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FIG.  7.  (left)  Angle  between  the  camera  line  of  sight  and  a  point  at  various  heights  above  the  waterline.  Once  the  angle 
becomes  negative,  a  target  of  the  depicted  heights  become  completely  obscured  by  the  horizon.  Displayed  another  way,  the 
right  hand  plot  gives  the  relationship  between  the  obscured  vertical  dimension  of  the  target  and  range. 


boat  being  incorrectly  identified  as  a  duck  boat).  The  crab  boat  also  resulted  in  an  improved  range  performance 
curve,  although  not  as  dramatic  as  for  the  other  two  ships. 

In  fact,  the  ranges  at  which  the  boats  could  be  reliably  identified  increased  nearly  to  the  point  at  which  they 
effectively  disappear  over  the  horizon.  The  camera  was  located  at  the  end  of  a  pier,  roughly  12  feet  above  the 
waterline.  Figure  (7a)  plots  the  angle  between  the  camera  line-of-sight  to  points  at  various  heights  above  the  water. 
Plotted  another  way  Figure  (7b)  shows  the  obscured  vertical  dimension  as  a  function  of  range.  The  duck  boat  is  closest 
to  the  waterline,  extending  only  a  few  feet  above  the  surface,  hence  this  boat  is  no  longer  visible  at  a  distance  of 
roughly  10km.  This  is  consistent  with  the  range  performance  curves  of  Fig.  (6).  Given  the  height  of  the  superstructure 
on  the  fishing  boat,  this  watercraft  is  the  last  to  become  fully  obscured  by  the  horizon,  hence  is  reliably  detected  at 
the  longer  ranges. 

It  is  also  important  to  note  that  the  confidence  intervals  for  the  estimated  curves  are  a  good  deal  more  narrow  than 
for  the  previous  set  of  experiments.  This  stems  from  the  significant  drop  in  misclassifications.  Whereas  observers 
could  detect  the  ships  reliably  with  the  previous  AWARE  camera,  the  non- uniformity  in  focus  across  the  FOV  was 
such  that  they  often  confused  one  ship  for  another,  resulting  in  the  larger  variance  in  observer  responses.  This  is  in 
contrast  to  the  current  curves  which  drop  suddenly  once  the  critical  detection  range  has  been  exceeded.  The  newer 
system  produced  significantly  fewer  misclassifications  and  only  in  a  very  narrow  range  of  distances. 


IV.  CONCLUSIONS 


A  second  generation  of  the  AWARE  visible  imager  was  described  and  its  performance  characteristics  reviewed.  In 
addition  to  occupying  a  smaller  footprint  in  terms  of  size  and  power,  the  system  performed  measurably  better  in  a 
maritime  range  performance  estimation  task.  We  were  able  to  quantitatively  show  a  several  kilometer  improvement 
in  identification  ranges  over  the  previous  incarnation  of  the  camera.  We  attribute  most  of  the  improvement  to  better 
focus  in  the  optics  across  the  field  of  view.  While  we  cannot  entirely  rule  out  weather  conditions  as  the  reason  for 
some  of  the  improvement,  all  data  were  collected  at  the  same  times  of  day,  (nearly)  the  exact  same  day  of  the  year, 
and  with  no  obvious  difference  in  weather  conditions  from  our  previous  data  set. 
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